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Preface 


The  highly  automated  flight  management  systems  in  today’s  aircraft  cockpit  barely  resemble 
the  stick  and  rudder  flight  controls  of  yesteryear.  Unfortunately,  current  automation  and 
associated  user  interfaces  are  not  always  intuitive  or  infallible.  Complex  cockpits,  faster,  more 
capable  aircraft,  airspace  saturation,  and  increasing  air  traffic  control  requirements  create  the 
environment  and  conditions  conducive  to  mode  confusion.  This  problem  is  both  real  and 
personal.  With  over  20  years  in  civilian  aviation  and  5000  hours  of  worldwide  military  flying 
experience  ranging  from  the  basic  Cessna  150  to  the  highly  automated  Boeing  757,  I  have  both 
witnessed  and  suffered  from  this  malady.  Reducing  mode  error  will  require  a  multifaceted 
approach  involving  designers,  trainers,  and  aircrew.  My  research  investigates  one  possible  piece 
of  the  total  solution  by  exploring  audible  attention  steps  that  might  quickly  and  properly  identify 
modal  problems  to  pilots  without  further  burdening  their  situation  awareness  with  more 
automated  clutter. 

The  seed  for  this  idea  was  planted  when  I  was  at  a  commercial  air  carrier’s  training  center 
learning  how  to  fly  the  B-757  for  the  Air  Force.  The  opportunity  for  organized  research 
presented  itself  by  way  of  an  Air  University  Research  topic  submitted  by  Air  Mobility 
Command.  I  owe  great  appreciation  to  my  Faculty  Research  Advisor,  Lt  Col  Steve  Kimbrell,  for 
his  repeated  admonitions  to  keep  the  research  manageable,  and  for  providing  the  needed  focus  to 
sufficiently  narrow  the  scope  of  my  research. 
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Abstract 

The  application  of  computer  technology  in  modern  cockpits  has  resulted  in  sophisticated 
automation  that  has  created  situations  of  mode  confusion  where  the  pilot  is  uncertain  about  the 
status  or  behavior  of  cockpit  automation.  Based  on  current  levels  of  cockpit  automation, 
classifications  of  mode  confusion,  and  clinical  knowledge  concerning  human  cognitive  and 
attentive  processes,  could  an  audible  attention  step  help  mitigate  unrecognized  mode  error?  The 
Software-Hardware-Environment-Liveware  model  forms  a  framework  for  the  analysis  of 
government  and  academic  research  concerning  pilot  automation  experiences  and  use,  cognitive 
models,  information  and  decision  processing,  and  the  auditory  attention  channel.  Pilot 
experiences  and  several  aircraft  accidents  suggest  that  mode  error  is  both  common  and 
potentially  dangerous  enough  to  warrant  attempts  at  mitigation.  Studies  indicate  that  the 
monitoring  requirement  levied  by  automation  lowers  pilot  system  situational  awareness  without 
providing  sufficient  or  proper  feedback.  Operators  can  also  suffer  from  cognitive  lockup  and  task 
channeling,  which  makes  attention  diversion  difficult.  An  auditory  input  might  provide  an 
effective  attention  step  if  it  demands  appropriate  attention,  provides  situation  reporting,  and 
offers  problem  guidance.  These  requirements  might  be  fulfilled  if  the  content  is  predictive, 
informational,  localized,  properly  timed,  and  the  theories  of  effective  auditory  characteristics 
proposed  by  Human  Factor’s  research  are  considered. 
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Part  1 


Introduction 

The  twentieth  century  has  witnessed  the  rapid  development  and  marriage  of  two  momentous 
technologies:  the  aircraft  and  the  computer.1  Today,  highly  automated  aircraft  make  the  pilot 
more  of  a  manager  than  an  active  flyer.  Ergonomic  design  has  not  kept  pace,  especially  in  the 
man-machine  interface.'  This  has  produced  situations  of  mode  confusion  where  the  pilot 
misunderstands  the  behavior  of  cockpit  automation.  Usually,  the  result  is  benign  inconvenience, 
but  it  has  occasionally  ended  in  tragedy. 

The  Research  Question.  There  are  basically  two  approaches  to  reducing  human  error.4  The 
first  is  to  determine  the  underlying  reason  for  the  problem  and  then  correct  it.  For  the  conditions 
discussed  in  this  paper,  this  approach  is  necessarily  equipment  specific  and  therefore  might  entail 
a  change  in  design.5  The  second  approach  is  to  mitigate  the  problem  by  quickly  and  properly 
identifying  the  situation  to  the  pilot  for  diagnosis  and  action.  This  identification  step  must 
address  the  most  common  mode  confusions  to  be  useful,  fit  within  the  pilot’s  cognitive 
framework  to  be  effective,  and  demand  sufficient  attention  to  be  correctly  recognized. 
Traditional  cockpit  designs  use  visual  and  auditory  inputs  to  perform  the  alert  function. 
Therefore,  based  on  current  levels  of  cockpit  automation,  classifications  of  mode  confusion,  and 
clinical  knowledge  concerning  human  cognitive  and  attentive  processes,  could  an  audible 
attention  step  help  mitigate  unrecognized  mode  error? 
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Conceptual  Framework.  The  research  question  is  basically  one  of  ergonomics  since  this  is 
primarily  a  man-machine  interface  issue.  The  field  of  Human  Factors  has  developed  the 
Software-Hardware-Environment-Liveware  (SHEL)  model  to  conceptualize  these  types  of 
relationships.  To  facilitate  analysis,  this  model  forms  a  framework  that  breaks  the  situation  down 
into  four  arms.6 


Figure  1.  The  SHEL  Model7 

The  “Liveware”  at  the  center  of  the  cross  is  the  pilot.  The  other  parts  of  the  model  constitute 
the  situation  that  surrounds  the  Liveware.  The  pilot  interacts  with  and  gets  feedback  from  each  of 
these  four  parts  (represented  by  arrows  going  both  directions)  thus  forming  four  “arms”.  The 
Software  arm  deals  with  procedures  and  computer  programs.  Checklists,  Llight  Operation 
Procedures,  and  Llight  Management  System  (LMS)  software  logic  and  internal  decision 
algorithms  fall  into  this  category.  The  Hardware  arm  covers  cockpit  layout  and  controls  such  as 
displays  and  the  types  and  placement  of  alerts  (caution  lights,  speakers,  fire  bells).  These  two 
arms  will  be  analyzed  first  by  examining  pilot’s  perceptions  about  cockpit  automation,  and  then 
investigating  the  possible  consequences  of  mode  confusion  in  order  to  determine  if  they  present 
potentially  dangerous  enough  situations  to  warrant  an  attention  step. 

The  next  arm,  Environment,  is  concerned  with  the  mental  and  physical  stresses  imposed  on 
the  Liveware.  The  mental  aspect  is  addressed  with  an  analysis  of  the  cognitive  approaches  pilots 
use  to  deal  with  cockpit  attention  and  workload  requirements.  The  physical  side  is  discussed  with 
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Human  Factors  studies  that  explore  the  most  effective  timing  and  type  of  auditory  attention 
steps.  Because  this  is  a  man-machine  interface  issue,  the  Liveware  arm  is  only  considered  as  it 
relates  to  workload  disturbances.  Examples  include  Air  Traffic  Control  (ATC)  communications 
and  interactions  with  other  crewmembers.  These  not  only  affect  mental  performance  but  also 
give  clues  as  to  the  effectiveness  of  auditory  inputs.  Finally,  each  arm  of  the  SHEL  model  is 
reassembled  to  provide  specific  recommendations  on  auditory  formats  that  might  effectively 
serve  as  attention  steps  for  error  mitigation. 

Research  Scope.  Efforts  at  reducing  mode  confusion  are  receiving  government  and 
academic  attention  at  the  international  level.8  This  research  is  but  a  small  part  of  the  larger  effort 
and  is  limited  to  exploring  auditory  inputs  for  error  mitigation.  Specific  system  or  cockpit 
redesign  or  the  use  of  other  sensory  channels  is  not  addressed.  Similarly,  cost  factors,  specific 
aircraft  implementation  feasibility,  or  design  criteria  are  not  included. 


Notes 

1  Raja  Parasuraman,  and  Victor  Riley,  “Humans  and  Automation:  Use,  Misuse,  Disuse, 
Abuse,”  Human  Factors  39,  no.  2  (June  1997):  230. 

2  P.  Douglas  Arbuckle,  et  al,  "Future  Flight  Decks,"  Proceedings  of  the  21st  Congress 
International  Council  of  the  Aeronautical  Sciences,  Melbourne,  Australia,  13  -  17  September 
1998,  10. 

Asaf  Degani,  Michael  Shafto,  and  Alex  Kirlik,  "Modes  in  Automated  Cockpits:  Problems, 
Data  Analysis,  and  a  Modeling  Framework,"  Proceedings  of  36th  Israel  Annual  Conference  on 
Aerospace  Sciences,  Haifa,  Israel,  1996.  n.p.;  online,  Internet,  4  October  1999,  available  from 
http://olias.arc.nasa.gov/publications/degani/modeusage/  modes2.html. 

4  Frank  H.  Hawkins,  Human  Factors  in  Flight ,  ed.  Harry  W.  Orlady,  2nd  ed.,  (Brookfield, 
Vt.:  Ashgate  Publishing  Co.,  1993),  50. 

5  R.  Onken,  Functional  Development  and  field  test  of  CASSY  -  a  knowledge-based  cockpit 
assistant  system,  AGARD  Fecture  Series  200:  Knowledge-Based  Functions  in  Aerospace 
Systems,  Quebec,  Canada:  Canada  Communication  Group,  November  1995,  4-3. 

6  Hawkins,  22. 

7  Ibid. 

8  Parasuraman,  250. 
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Part  2 


Background:  The  Pilot-Cockpit  Automation  Relationship 


Anecdotal  evidence  suggests  that  where  there  is  a  man-machine  interface  of  any  complexity, 
there  is  modal  confusion. 1  The  first  question  is  whether  pilot  misunderstanding  of  automation  is 
of  a  significant  enough  consequence  to  warrant  efforts  at  mitigation.  If  so,  then  what  type  of 
mode  confusion  either  currently  is  or  holds  the  most  potential  to  be  the  greatest  threat  to  aviation 
safety?  Due  to  the  serious  nature  of  aircraft  safety  in  terms  of  lives  and  financial  cost,  there  has 
been  ongoing  research  into  the  actual  effects  of  mode  confusion.  These  studies  help  define  the 
problem  and  level  of  concern  by  investigating  pilot  interactions  with  the  Hardware  and  Software 
aspects  of  the  SHEL  model. 


Pilot  Perceptions 

Addressing  the  perceptions  of  pilots  that  fly  automated  aircraft  forms  a  starting  point  for  a 
mode  confusion  analysis.  The  Royal  Air  Force  Institute  of  Aviation  Medicine  and  the  University 
of  Illinois/NASA  (National  Aeronautics  and  Space  Administration)  Ames  both  studied  the  pilot 
attitudes  of  443  and  168  crewmembers  respectively  who  flew  highly  automated  glass  aircraft 
(A320,  B747-400,  B757/767).2  The  results  were  strikingly  similar.  Both  studies  found  that 
current  automation  does  not  provide  sufficient,  proper  feedback.  Audible  or  visual  alerting  was 
also  missing,  making  failures  difficult  to  comprehend  and  correct.  Additionally,  increased 
monitoring  requirements  lead  to  complacency  and  loss  of  basic  scanning  flows  and  flying  skills. 
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Participants  also  felt  that  automation  actually  increases  workload  during  high  task  phases.  This 
is  due  to  the  progressive  nature  of  flying.  Pilots  do  not  have  the  luxury  of  “pulling  over”  to  the 
side  of  the  road  like  in  an  automobile  to  figure  out  a  situation.  The  aircraft  continues  to  move  on 
a  three-dimensional  path  even  if  the  pilot  is  not  aware  of  the  current  aircraft  configuration  or  is 
not  prepared  for  future  tasks  and  situations.  The  myriad  number  of  situations  possible  require 
that  an  automated  system  must  be  capable  of  complex  programming,  which  in-tum  requires  time 
and  thought.  Thus  the  actual  workload  may  increase,  especially  with  rapid  ATC  commands  in  a 
congested  terminal  area.  Without  audible  alerts,  the  pilot  may  be  unaware  of  system  status. 

The  second  study  found  that  over  80  percent  of  respondents  experienced  aviation  surprises, 
of  which  39.3  percent  of  these  never  discovered  why  and  42.9  percent  had  to  have  it  explained.4 
These  surprises  took  two  forms;  automation  failed  to  take  the  expected  action,  or  it  took  an 
uncommanded  unexpected  action.5  Researchers  concluded  that  automation  surprises  are  due  to 
poor  mental  models,  low  system  observability  (no  displays  or  alerts),  and  a  highly  dynamic 
situation  where  pilots  look  for  commanded  actions  instead  of  actual  behavior.  Recommendations 
included  increasing  auditory  and  visual  feedback  to  promote  parallel  information  processing.6 

Automation  Surprises 

As  was  discussed,  current  automation  has  the  potential  to  surprise  pilots  with  its  behavior. 
But  just  how  consequential  are  these  surprises?  In  1995  the  Federal  Aviation  Administration 
Aircraft  Certification  Directorate  commissioned  a  study  on  modem  glass  cockpit  automation 
interfaces  in  response  to  several  aviation  accidents  blamed  on  automation  confusion.  The  global 
commercial  jet  airline  loss  rate  has  remained  constant  since  1975,  even  though  the  number  of 
departures  has  almost  tripled.  From  1987  to  1996,  71.7  percent  of  the  71  percent  of  accidents 
with  known  causes  were  blamed  on  aircrew.9  The  study  expressed  concern  about  the  level  of 
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pilot  understanding,  use,  and  modal  awareness  of  automation,  stating  that  when  the  pilot  is  not 
sufficiently  engaged,  he  enters  a  “hazardous  state”  of  awareness.10  They  do  not  feel  the  problem 
is  procedural  since  20  years  has  not  seen  a  drop  in  the  aircraft  loss  rate  even  with  airline 
emphasis  on  standardization.  They  recommend  task-based  displays,  real-time  system  fault 
management,  and  higher  pilot  knowledge  levels  concerning  automation.1 1 

An  altitude  deviation  is  one  common  consequence  of  mode  confusion  that  contains  accident 
potential.  NASA  Ames  researched  500  reports  from  the  Aviation  Safety  Reporting  System 
(ASRS)  and  found  this  to  be  the  most  reported  problem.  In  glass  cockpits,  Air  Traffic  Control 
detected  half  the  occurrences,  while  for  flight  crews,  the  altitude  alerter  only  prompted  10 
percent  of  the  recoveries.  The  researchers  recommend  better  and  redundant  feedback  loops  for 
anomaly  detection,  as  well  as  new  system  designs.12 

Modes  in  Human- Automation  Interaction 

It  has  been  established  that  mode  confusion  occurs  and  can  have  significant  consequences, 
but  how  often  are  errors  committed,  and  of  what  type?  Between  1997  and  1998  a  University  of 
Texas  Research  Team  conducted  Line  Operations  Safety  Audits  on  184  crews  with  three  airlines. 
They  found  that  68  percent  of  crews  committed  at  least  one  error,  the  largest  of  which  (31 
percent)  was  associated  with  automation.  Most  were  a  failure  to  verify  settings  (65  percent). 
Incorrect  switch  or  mode  selection  constituted  21  percent.  Over  one-half  of  errors  were  not 
corrected  (53  percent),  36  percent  were  fixed,  and  11  percent  were  mismanaged.  A  majority  of 
outcomes  were  inconsequential  (85  percent),  although  12  percent  resulted  in  an  undesired 

13 

aircraft  state  with  altitude  deviations  being  the  most  common. 

A  NASA  Ames  study  conducted  on  30  flights  aboard  B757/767  aircraft  helps  categorize 
mode  confusion.  One  of  the  pilot’s  functions  is  to  modify  and  monitor  the  automation’s  interface 
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and  control  modes.  These  actions  range  from  manual,  to  semi-automatic,  to  fully  automatic  in 
nature.  Automation  behavior  is  intuitive  if  it  is  represented  in  a  display  to  the  crew,  or  unintuitive 
if  it  is  based  on  an  unknown  internal  value  or  reference.14  The  researchers  categorize  mode 
confusion  into  two  areas.  The  first  is  misidentification  of  automation  behavior  when  its  actual 
behavior  is  different  from  what  was  expected.  The  second  is  where  the  pilot  acts  on  the 
assumption  that  the  machine  is  in  one  mode,  when  it  is  really  in  another.15 

Discussion 

Analyses  of  pilot’s  perceptions  of  highly  automated  cockpits  reveal  that  they  feel  their  job  is 
more  systems  management  than  actual  hands  on  flying,  which  erodes  their  scanning  and  basic 
flying  skills.  All  agree  that  increased  diversified  feedback,  auditory  and  visual,  is  needed  to 
ascertain  automation  status  and  behavior.  Even  though  airlines  have  made  concerted  procedural 
standardization  efforts,  the  specter  of  several  recent  automation  related  accidents  combined  with 
a  fairly  constant  rate  of  pilot  induced  aviation  accidents  since  1975  infer  that  procedural  change 
alone  has  not  substantially  impacted  mode  confusion.  There  is  also  ample  evidence  in  the  NASA 
ASRS  database  of  mode  confusion  that  contributed  to  altitude  deviations.  In  addition,  studies 
show  that  a  majority  of  crews  make  automation  errors  on  most  flights,  most  of  which  are  not 
understood  or  corrected,  with  10  percent  resulting  in  an  unintended  aircraft  configuration. 
Studies  of  mode  usage  reveal  that  confusion  falls  into  basically  two  camps;  misidentification  due 
to  expectations,  and  actions  based  on  incorrect  assumptions  about  the  engaged  mode. 

A  Catholic  University  of  America  study  on  how  operators  use  automation  provides  further 
support  for  these  observations.  They  found  that  designers  build  the  human  role  around  the 
automation,  leaving  the  operator  to  manage  the  resulting  system.16  They  assume  the  user  will 
operate  the  machine  in  a  certain  way,  yet  have  to  trust  them  to  use  their  own  judgement  in 
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dealing  with  diverse  situations.  Predicting  trust  and  use  of  automation  is  very  difficult,  thereby 

17 

exacerbating  the  problem  of  human-centered  design.  This  has  resulted  in  insufficiently 
designed  displays  and  feedback  mechanisms  that  actually  lower  operator  mode  awareness.  The 
monitoring  function  removes  the  operator  from  the  situation.  The  further  removed  the  pilot,  the 
more  the  attention  step  must  overcome.19  Suggested  alerts  are  those  that  offer  prediction,  system 

perception,  and  sufficient  time  to  respond  to  high  consequence  conditions  (not  just  tell  of  a 

20 

dangerous  situation  that  already  exists).” 

These  discussions  have  established  that  mode  confusion  exists,  but  do  not  specify  what 
types  of  confusions  warrant  a  separate  attention  step.  For  this  discussion,  mode  confusion  will  be 
defined  as  a  human  misunderstanding  of  an  automated  mode,  whether  it  is  an  unrecognized  fault 
or  an  unrecognized  behavior.  This  may  be  broken  down  into  designed  behavior ,  which  is  either 
an  active  behavior  (an  action,  mode  switch  or  capture)  or  a  passive  behavior  (did  not  act,  switch, 
or  capture),  both  determined  by  internally  referenced  values  or  states.  This  type  of 
misunderstanding  of  this  area  can  be  overcome  with  redesign  or  operator  knowledge,  and  is  not 
addressed  here.  The  second  part  is  error ,  which  is  the  failure  of  the  automation  to  behave  in  the 
designed  active  or  passive  manner  (a  mode  switch  or  capture,  or  a  specific  action)  due  to  a 
temporary  anomaly  or  internal  fault.  This  failure,  such  as  the  conditions  leading  to  an  altitude 
deviation,  or  a  missed  localizer  (LOC)  or  glideslope  (GS),  is  the  situation  that  this  paper  is 
addressing  for  an  attention  step. 

It  is  interesting  to  note  that  the  above  studies  mention  specific  system  deficiencies  such  as 
insufficient  feedback  and  display  of  automation  faults.  They  allude  to  the  idea  that  proper 
audible  alerts  might  overcome  these  design  shortcomings.  The  next  two  sections  explore  what 
types  of  audible  alerts  might  be  most  effective  for  mode  error. 
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Part  3 


Cognitive  Models  and  The  Auditory  Channel 


The  Environment  and  Liveware  arms  are  addressed  by  analyzing  pilot  cognitive  processing 
models  to  ascertain  the  best  timing  for  an  attention  step.  In  addition,  a  review  of  clinical  auditory 
research  provides  clues  to  determine  what  might  be  the  most  effective  auditory  input. 

Human  Cognitive  Information  Processing  Models 

The  University  of  Illinois  study  cited  incorrect  mental  models  as  one  aspect  leading  to 
automation  surprises.  The  cognitive  models  that  pilots  employ  to  process  information  and 
prioritize  workload  provide  insight  into  when  an  audible  attention  step  might  be  most  effective. 

Workload 

The  Time/Intensity  Relationship.  As  previously  discussed,  the  very  nature  of  flight  is 
dynamic  and  temporary.  Intuitively,  it  seems  that  the  less  time  that  is  available  to  complete  a 
task,  the  less  likely  it  is  that  a  correct  solution  will  be  achieved.  A  University  of  Toronto  study 
investigated  the  relationship  between  a  time -based  factor  and  an  intensity-based  factor 
(information  load)  to  determine  a  quantitative  workload  and  to  test  human  processing  capacity.  A 
simulated  ATC  environment  was  used  to  construct  a  complex  cognitive  task. 1  As  expected,  time 
pressure  is  more  of  factor  in  determining  performance  than  information  load."  Therefore,  on 
every  flight  pilots  automatically  face  a  situation  where  performance  is  challenged. 
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Task  Complexity,  Time,  and  Monitoring.  Previous  research  suggests  that  humans  do  not 
monitor  for  failure  very  efficiently.  Additionally,  dependence  on  automation  may  desensitize 
pilots  to  other  inputs  that  do  not  agree  with  the  automation.  Another  Catholic  University  of 
America  study  tested  subjects  in  a  realistic  flight  simulation  package  where  they  performed 
single  and  multiple  tasks  while  monitoring  for  failures.4  The  researchers  found  that  vigilance 
decreases  with  time,  and  that  failures  are  noticed  more  often  when  subjects  are  manually 
involved  in  the  task  that  failed,  as  opposed  to  just  monitoring  automation.5  However,  if  manually 
involved  in  a  task  separate  from  the  failed  system,  monitoring  is  less  efficient.6  This  research 
highlights  monitoring  efficiency  as  a  critically  important  human  limitation.  This  is  reflected  in 
the  antidotal  observation  (origin  unknown)  that  flying  is  basically  hours  of  sheer  boredom 
followed  by  moments  of  stark  terror.  Automation  supplies  the  boredom  by  imposing  a 
monitoring  requirement,  then  presents  terror  with  an  unforeseen  situation  that  resulted  from 
unrecognized  mode  error  due  to  insufficient  alerting. 

The  Model  Degree  of  Automation  for  Predicting  Mental  Loads.  Automation  does  not 
always  reduce  workload  since  the  nature  of  the  task  and  the  required  monitoring  function  has  to 
be  considered.  The  Delft  University  of  Technology  in  the  Netherlands  worked  on  formulating  a 
criteria  (a  Model  Degree  of  Automation)  for  determining  what  functions  should  be  automated 
based  upon  operator  manual  task  demand  (workload),  number  of  automated  functions  (task 
criticality)  and  mental  loads  (mental  calculations).  They  found  that  there  is  a  break-even  point 
for  automation,  where  more  actually  lowers  performance.  This  is  due  to  lower  situational 
awareness  if  an  operator  is  required  to  takeover  higher  level  functions,  a  loss  of  skill,  decreased 
vigilance  with  time  (lower  system  awareness),  and  the  idea  that  the  monitoring  task  itself 
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imposes  a  mental  workload.  Additionally,  depending  upon  the  nature  of  the  task,  mental 
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workload  may  not  decrease  with  automation  use  since  the  user  may  make  backup  mental 
calculations  to  reinforce  trust  in  the  system.9  Researchers  recommend  balancing  human 
involvement  (increased  situation  awareness)  with  automation  efficiency.10 

Information  and  Decision  Processing 

Decision  Processing  Model.  One  facet  of  mental  workload  is  the  method  of  information 
processing.  NASA  Ames  conducted  a  series  of  studies  on  professional  pilots  from  two  major  US 
airlines  to  determine  what  influenced  their  decision  making  process,  and  if  experience  was  a 
factor.  They  found  that  pilots’  balance  perceived  risk  with  time  available,  only  considering 
multiple  options  if  time  allows.11  The  time-limiting  nature  of  flight  sometimes  requires  the  pilot 
to  act  (or  lockup,  described  below)  either  impulsively  based  on  a  hunch  or  instinctively  based  on 
experience.  The  military,  police,  and  pilots  are  just  a  few  occupations  that  use  repetitive  training 
(experience)  that  is  designed  to  elicit  automatic  responses  when  faced  with  certain  situations. 
Even  driving  an  automobile  can  require  the  driver  to  make  split  second  risk/time  decisions,  such 
as  swerving  or  slamming  on  the  brakes  to  avoid  an  unexpected  obstacle  depending  upon  other 
threats  (oncoming  car). 

Pilot  Information  Categorization.  In  order  to  take  the  most  appropriate  action,  decisions 
require  a  task  prioritization  scheme  for  categorizing  information.  A  NASA  Ames  study  on  how 
pilots  categorized  information  involved  52  pilots  from  eight  different  airlines.  “  The  researchers 
found  that  pilots’  rank  information  as  strategic  or  tactical,  and  use  the  source  (where  it  originates 
from)  and  destination  (where  it  is  displayed),  to  get  clues  about  content.  Generally,  information 
is  grouped  as  aviate,  navigate,  communicate,  or  systems  administrate,  where  safety  is  first, 
followed  by  time  critical  tasks.14  This  decision  scheme  is  central  to  military  flight  training,  and  is 
stressed  for  every  situation  faced.  The  plane  never  stops  flying  and  must  be  controlled  (aviate),  it 
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will  proceed  on  a  certain  path  (navigate),  others  must  know  of  the  situation  in  order  to  help  or  get 
out  of  the  way  (communicate),  and  the  aircraft  must  be  managed  to  rectify  or  mitigate  the 
problem  and  configure  for  landing  (system  administrate).  Only  a  safety  of  flight  issue  should 
upset  this  order.  A  radio  call  to  other  aircraft  and/or  ATC  in  order  to  request  landing  priority  or 
to  clear  the  emergency  aircraft’s  flight  path  is  one  such  example.  Researchers  also  discovered 
that  information  should  be  presented  in  multiple  channels  (auditory  and  visual),  and  that  scaling 
(size)  and  clustering  (grouping)  help  in  prioritization.15  Additionally,  automation  implicitly  adds 
system  knowledge  requirements  to  understand  its  operation  and  behavior.16 

Disturbances  and  Determinates.  For  this  paper,  the  Liveware  arm  relates  to  human 
intrusion  and  disturbance  of  the  cockpit  workload.  This  could  be  by  ATC,  another  crewmember, 
or  even  a  passenger.  Previous  studies  suggest  that  people  tend  to  focus  on  one  problem  area  to 
the  exclusion  of  other  systems,  working  sequentially  through  problems.  The  TNO  Human 
Factors  Research  Institute  in  the  Netherlands  postulate  that  operators  intervene  in  automatic 
functions  only  to  adjust  or  fix  malfunctions,  ignoring  other  systems  until  the  situation  becomes 
too  complex  to  handle  (raising  the  possibility  of  an  accident).  Their  study  had  subjects  perform 
a  monitoring  function  that  was  affected  by  a  number  of  disturbances  that  required  time  limited 
diagnosis  while  operating  under  time  constraints.19  They  found  that  a  monitoring  activity  stops 
during  diagnosis,  with  resistance  to  intrusion  regardless  of  other  disturbances.  Eventually  this 
results  in  cognitive  lockup  where  monitoring  and  diagnosis  is  ineffective.”  This  emphasizes  the 
importance  of  time,  level  of  monitoring/involvement  (situation  awareness),  and  especially  proper 
mental  models  (decision  schemes)  to  cope  with  mode  errors. 

A  critical  interface  that  falls  under  both  the  Liveware  and  Environment  arms  is  the 
relationship  between  the  pilot  and  ATC.  These  communications  have  a  direct  impact  on  the 
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cognitive  processes  employed  in  the  cockpit.  A  study  conducted  by  NASA  attempts  to  discern 

how  flight  control  strategies  in  complex  cockpits  is  affected  by  ATC.  The  criteria  are  divided 

into  clearance  type,  predictability,  amount  of  clearances,  time  constraints,  and  use  of  automation. 

They  found  that  the  more  intense  the  environment  (larger  numbers  of  commands)  and  the  higher 

the  workload  (less  time  to  act),  the  less  automation  is  used.  Initially  this  seems  counterintuitive, 

although  it  is  hypothesized  that  manual  control  allows  faster  response  times  and  lower  mental 
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loads  since  backup  calculations  and  automation  knowledge  and  monitoring  is  not  required. 

The  Cockpit  as  a  Distributed  Cognitive  System.  A  study  by  NASA  Ames  analyzed  ten 
altitude  deviation  reports  in  the  ASRS  by  considering  the  entire  system  of  aircraft,  pilots,  and 
controllers  as  one  cognitive  structure.  This  is  one  example  where  several  agencies  are  involved 
in  maintaining  safety.  Errors  are  considered  as  either  a  deviation  from  expected  behavior  or 
expected  information.  Hypothesizes  that  for  altitude  changes  there  are  three  cognitive  tasks: 
intention,  specification,  and  execution.  Distractions  have  the  greatest  impact  on  execution  and 
cross-checking.  The  altitude  alerting  function  is  not  a  predictive  alert,  but  is  a  condition  alert, 
which  does  not  necessarily  preempt  an  altitude  bust  when  the  execution  and  cross-check  actions 
break  down.  The  researchers  recommend  better  cockpit  cross-check  procedures,  and  an 
intelligent  audible  alert  that  provides  sufficient  time  for  action.  The  characteristics  of  this  type 
of  alert  are  discussed  below. 

The  Auditory  Attention  Channel 

In  the  cockpit  environments  described,  the  correct  type  of  audible  attention  step  must  be 
used  to  maximize  the  probability  that  it  will  demand  the  proper  attention  in  sufficient  time  to 
allow  diagnosis  and  action.  Clinical  studies  on  auditory  input  effectiveness  provide  possibilities 
for  this  attention  step. 
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System  Sensitivity  versus  Dependence  of  Alarm  Monitored  Information  Sets 

Cockpit  automation  may  monitor  or  control  several  subsets  of  systems  for  the  pilot  and 
provide  an  alert  for  an  anomaly.  The  question  for  designers  is  how  to  achieve  maximum 
relational  efficiency  of  human  and  machine  monitoring  functions.  In  a  University  of  Dayton 
study  participants  performed  two  tasks  concurrently,  with  one  being  a  signal  detection  task.” 
The  conditions  to  be  monitored  were  either  dependent,  partially  dependent  (overlapping),  or 
independent  of  automated  functions.”  The  most  efficient  situations  are  where  the  man-machine 
observations  are  de-correlated,  which  provides  the  most  information  to  the  operator.  The  more 
the  monitoring  jobs  overlap,  the  less  efficient  the  relationship.  They  also  found  that  alarms  do 
raise  awareness,  but  they  increase  the  operator  workload  depending  upon  how  much  decoding 
must  be  done  to  achieve  proper  understanding.  Although  no  specifics  were  given,  they  also  felt 
that  the  alarm  effect  should  be  maximized  to  avoid  operator  desensitization.26  This  aspect  of  the 
alert  is  explored  below.  The  issue  with  this  study  is  their  definition  of  efficiency,  which  was 
directly  related  to  the  amount  of  information  available  to  the  operator.  As  has  been  discussed, 
excessive  information  may  cause  cognitive  lockup.  Additionally,  totally  de-correlated 
monitoring  tasks  place  the  pilot  in  the  situation  of  being  out  of  the  loop,  which  lowers  situational 
or  system  awareness  making  timely  intervention  difficult. 

Stepping  out  of  pure  clinical  research  into  what  is  occurring  in  current  aircraft  design,  in 
1997  the  United  States  Army  Aeromedical  Research  Laboratory  reviewed  the  current  state  of 
aircraft  multifunction  display  and  control  systems  design.  Although  more  concerned  with  visual 
displays,  they  do  suggest  that  auditory  warning  and  advisory  systems  could  take  a  more  complex 
approach,  including  providing  a  more  three-dimensional  (surround  sound)  input  for  threat 
localization.  Messages  should  be  short  and  to  the  point,  with  sufficient  time  allowed  to  interpret 
the  auditory  input.”  This  idea  of  sound  localization  has  been  researched  by  the  Air  Force  for  use 
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in  helmet  mounted  displays.  The  time-for- interpretation  idea  is  discussed  in  the  NASA  Ames 
altitude  deviation  study  and  the  Catholic  University  of  America  research  in  the  form  of 
suggesting  predictive  warnings  instead  of  real-time  situation  alerts. 

Sound  Localization  and  Content 

Sound  localization  as  discussed  in  the  Army  paper  was  researched  by  the  USAF  Research 
Laboratory  at  Wright  Patterson  AFB.  Localization  presents  an  auditory  signal  either  from  or 
directed  to  a  specific  geographic  location  with  the  intention  of  allowing  the  listener  to  connect  a 
sound  with  a  point  in  space.  Non-localized  signals  are  for  conveying  information  without 
drawing  or  directing  attention.  For  example,  for  a  target  directly  to  the  pilot’s  right,  the  alert  of 
“target”  from  an  overhead  speaker  does  not  provide  direction,  while  the  same  alert  from  a 
speaker  at  that  location  or  the  alert  of  “target:  right”  provides  direction.  Although  this  work  was 
done  to  compare  localized  and  non-localized  auditory  inputs  for  visual  target  acquisition  and 
workload  in  Helmet  Mounted  Displays,  the  idea  could  be  extended  to  other  cockpit  situations.” 
They  found  that  localized  auditory  inputs  greatly  increase  target  acquisition,  reduce  head 
movements,  and  lower  workload.  The  requirement  to  monitor  objects  outside  the  field  of  view  is 
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also  reduced.” 

A  study  done  by  the  Massachusetts  Institute  of  Technology  reinforces  these  findings.  They 
researched  auditory  alarm  content  and  direction  for  their  effectiveness.  They  found  that 
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spatially  located  sounds  and  informational  content  both  increase  visual  target  acquisition  speed. 
However,  large  amounts  of  information  or  multiple  auditory  warnings  reduce  speed  and 
accuracy.  '  Most  people  with  normal  hearing  can  locate  a  sound’s  true  location  horizontally  (on 
one  plane)  to  7  degrees  plus  or  minus  3  degrees.  The  researchers  found  that  six  speakers  spread 
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evenly  (horizontally)  is  sufficient,  with  any  more  being  detrimental. 
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Figure  2.  Horizontal  (Planal)  Geographic  Sound  Location  Ability 


Effectiveness  of  Aural  Interruption 

Cockpits  make  extensive  use  of  the  visual  sense,  with  auditory  inputs  used  for 
communication,  warnings,  cautions,  and  advisories.  If  the  intent  is  to  provide  an  attention  step, 
what  channel  might  garner  attention  more  efficiently?  The  NASA  Langley  Research  Center 
studied  the  influence  of  interruption  on  the  pilot  performance  of  fourteen  commercial  airline 
pilots.  The  original  study  researched  attention  steps  for  datalink  communications.  Both  visual 
and  auditory  interruptions  were  tested.34  They  found  that  aural  interruptions  are  more  effective  at 
gaining  attention  and  response  than  visual  inputs.  However,  the  introduction  of  another  alert  in 
the  same  channel  does  not  receive  the  same  attention.  The  reason  is  one  of  task  focus,  with 
auditory  engagement  (communications)  harder  to  disrupt  once  the  operator  is  already  engaged  in 
another  task.  The  disturbances  study  by  the  TNO  Human  Factors  Research  Institute  in  the 
Netherlands  also  supports  this  finding. 

Effective  Auditory  Inputs 

Historical  research  from  the  field  of  Human  Factors  can  provide  valuable  information  on  the 
human  auditory  sense.  Human  response  to  sensory  input  has  been  studied  for  years,  with  insight 
on  effective  auditory  input  readily  available.  This  research  delineates  three  fundamental 
principles  for  the  design  of  auditory  alerting  signals.  It  must  demand  attention,  report  the  nature 


of  the  condition,  and  provide  a  guide  to  the  appropriate  action.  To  demand  attention,  the  signal  to 
noise  ratio  must  be  such  that  the  input  will  be  intelligible.36  The  alert  needs  to  be  reliable,  with 
the  correct  situational  threshold  to  avoid  becoming  so  commonplace  as  to  be  ignored  or  turned 
off.  To  report  the  condition  in  an  understandable  manner,  common  words  and  phrases  must  be 
used  to  provide  expectancy,  and  they  cannot  overlap  with  other  alerts.  Designers  must 
remember  that  humans  can  absorb  large  amounts  of  sensory  data,  but  have  small  short-term 
memory  capacity,  so  the  messages  must  be  brief.  The  ideas  of  localization  and  content  that 
have  already  been  discussed  can  help  fulfill  the  guidance  principle. 

The  application  of  these  principles  married  with  the  four  level  safety  decision  scheme 
previously  discussed  has  resulted  in  four  categories  of  alerts.  The  first  level  is  safety  related 
(impending  stall,  terrain  or  traffic  warnings),  which  corresponds  to  the  most  important  decision 
aspects  of  “aviate”  or  “navigate”.  The  second  level  concerns  aircraft  configuration  (flaps,  gear), 
which  is  tied  to  either  “aviate”  or  “system  administrate”.  The  third  level  relates  to  system  status 
(fuel,  engines),  which  is  “system  administrate”.  The  lowest  level  is  communications,  such  as  an 
intercom  “ding”  or  an  ATC  datalink  light  or  radio  call.40  Although  the  last  two  alerts  appear  to  be 
out  of  order  with  respect  to  the  decision  scheme,  it  must  be  looked  at  from  both  a  pilot  and  a 
designer  perspective.  The  scheme  guides  the  pilot’s  actions,  so  the  designer  should  only  include 
a  system  alert  that  identifies  an  important  status  or  malfunction  that  either  currently  is  or  might 
deteriorate  into  a  safety  of  flight  issue.  The  alert’s  characteristics  (fire  bell  or  advisory  message) 
help  the  pilot  rank  its  importance  to  the  situation.  Communication  alerts  are  prompts  for 
attention,  either  from  another  crewmember  or  from  ATC.  If  the  message  is  of  sufficient 
importance,  the  individual  will  demand  attention  by  voice,  not  by  alert.  Therefore,  the  safety 
reason  of  the  system  alert  ranks  higher  than  the  information  reason  of  the  communication  alert. 
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Human  Factor’s  research  hypothesizes  that  there  are  three  functions  humans  use  when 
receiving  auditory  inputs.  These  functions  (detection,  relative  discrimination,  and  absolute 
identification)  work  most  efficiently  when  the  input  is  short,  simple,  does  not  have  to  be 
remembered  for  long  periods,  and  relates  to  an  action  required  immediately.41  In  addition, 
auditory  inputs  are  more  effective  when  complemented  with  the  visual  sense.  Thus,  the  pilot 
decision  scheme,  the  reason  for  the  alert,  and  human  physiology  all  should  determine  the 
intensity  and  type  of  alert.  Other  factors  such  as  the  use  of  headsets  or  earplugs  should  also  be 
considered.  Traditionally,  alerts  range  in  form  from  an  emergency  fire  bell  and  associated 
warning  light  to  an  informational  interphone  “ding”.  The  dynamic  and  constantly  changing 
nature  of  flight  coupled  with  increasing  system  complexity  presents  the  pilot  with  ever 
increasing  numbers  of  alerts.  For  example,  the  first  intercontinental  jet,  the  B707,  had  188  aural 
alerts.  The  modern  B747  has  at  least  455. 4j 

Discussion 

Research  indicates  that  workload  and  human  mental  capacity  may  be  a  function  of  intensity, 
which  is  information  load  over  time.  The  higher  the  load  and  shorter  the  time,  the  less  efficient 
(correct)  the  performance.  In  the  simplest  sense,  the  basic  idea  is  that  automation  will  lower 
workload.  Why  have  the  pilot  manually  control  the  ailerons,  rudders,  and  elevators  to  stay  on 
course  and  altitude,  turn  on  and  off  the  fuel  pumps,  control  the  engine  generator  frequencies, 
constantly  adjust  the  throttles  for  speed,  etc.,  etc.,  when  automation  can  just  let  him  watch?  This 
will  improve  the  efficiency  of  the  pilot  and  allow  time  for  higher  level  tasks.  However, 
depending  upon  the  nature  and  criticality  of  the  task  and  the  number  of  systems  automated, 
workload  may  not  decrease  as  much  as  originally  thought.  This  is  due  to  the  monitoring  task  that 
the  automation  should  receive  from  the  pilot.  The  very  fact  that  automation  exists  adds  yet 
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another  level  of  system  knowledge  that  the  pilot  needs  to  understand  the  machine,  adding  to  the 
task  load  that  monitoring  itself  imposes.  Not  only  does  the  pilot  have  to  understand  the  normal 
systems  (fuel,  electrics,  hydraulics,  engines,  flight  controls,  etc.),  but  he  also  has  to  comprehend 
the  system  controlling  the  systems,  determine  what  it  is  doing,  and  program  it  in  short  order. 

Studies  also  indicate  that  humans  do  not  monitor  well,  with  vigilance  decreasing  over  time. 
Automation  and  monitoring  further  remove  the  pilot  from  manual  tasks,  lowering  the 
situational/system  awareness  as  well  as  reduce  flying  and  scanning  proficiency  (if  there  is  less 
hands-on  practice).  All  of  these  factors  combine  to  raise  the  pilot’s  attention  threshold  making 
mode  error  recognition,  diagnosis  and  intervention  more  difficult. 

According  to  research,  pilots  categorize  information  according  to  a  risk-time  model,  only 
considering  additional  options  if  time  allows.  Safety  is  first,  in  the  general  order  of  aviate, 
navigate,  communicate,  or  system  administrate,  with  strategic  and  tactical  dimensions.  When  a 
disturbance  (alert,  ATC  communication)  occurs,  people  tend  to  focus  on  that  issue  to  the 
exclusion  of  other  duties  (to  the  point  of  resistance),  including  monitoring,  cross-checking,  and 
task  executions.  Even  additional  disturbances  may  not  receive  the  same  level  of  attention  until 
the  original  issue  is  resolved,  possibly  ending  in  cognitive  lockup.  Interestingly,  as  workload 
intensity  increases,  pilots  tend  to  use  less  automation,  preferring  to  accomplish  tasks  manually. 
This  is  possibly  for  quicker  response,  lower  mental  loads,  or  higher  focus  for  better  performance. 

The  situation  for  the  pilot  as  described  in  the  paragraphs  above  probably  oscillates  between 
the  extremes  of  total  disengaged  boredom  to  a  highly  focused  concentration  on  one  task  to  the 
exclusion  of  all  others.  The  entire  situational  spectrum  requires  a  specifically  designed  attention 
alert.  Research  indicates  that  audible  alerts  make  effective  attention  steps,  especially  when  used 
in  concert  with  or  in  support  of  visual  displays.  With  this  in  mind,  what  audible  alert  would  be 
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the  most  effective?  A  method  to  approach  this  question  lies  in  the  three  principles  for  alert 
design.  The  first  guideline  is  that  it  must  demand  attention.  Human  Factor’s  research  suggests 
that  it  must  rise  above  the  noise  to  be  intelligible  and  must  be  differentiated  from  the  myriad  of 
other  audible  alerts  that  are  used  on  modem  aircraft  to  be  recognized.  It  should  not  be  duplicative 
of  normal  pilot  tasks  to  the  point  of  becoming  a  confirmation  alert,  and  be  reliable  enough  so  as 
not  to  become  a  nuisance  to  be  ignored  during  high  workloads  or  times  of  focus  on  the  diagnosis 
of  other  system  malfunctions. 

The  second  principle  concerns  situation  reporting.  Research  suggests  that  it  should  contain 
common  language  in  short  message  format  since  short-term  memory  is  limited.  Alerts 
traditionally  fall  into  four  categories  that  range  from  safety  to  communications.  This  should 
determine  the  loudness  and  type  (bell,  voice,  muted  tone)  of  auditory  input.  Studies  repeatedly 
stress  that  the  informational  content  should  be  predictive  if  possible,  allowing  enough  time  for 
diagnosis  and  action.  This  is  as  important  for  the  disengaged  pilot  with  low  situational  awareness 
as  it  is  for  the  overworked  pilot  focused  on  one  task. 

The  third  principle,  guidance,  might  be  addressed  by  using  localization.  This  method 
reduces  reaction  and  acquisition  time.  Only  small  numbers  of  speakers  need  to  be  used  for 
general  direction  or  three-dimensional  effects.  Directing  the  pilot’s  attention  to  an  area  of  the 
cockpit  where  the  mode  error  is  or  has  occurred  (the  switch  or  display  location)  could  increase 
reaction  time  and  improve  the  informational  content. 
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Part  4 


Recommendations  and  Conclusion 

The  information  presented  in  the  Background,  Cognitive  Models,  and  Auditory  Channel 
sections  is  theoretical  in  scope.  This  part  narrows  and  synthesizes  these  ideas  by  providing 
examples  of  possible  real  world  applications  and  areas  for  future  consideration. 

Recommendations.  Table  1  presents  several  notional  scenarios  based  on  the  C-32A  (B757- 
200ER)  where  effective  auditory  alert  principles  are  applied.  These  errors  were  chosen  for  their 
possible  deadly  midair  or  ground  collision  potential.  Additionally,  the  autothrottle  (A/T)  failure 
example  could  result  in  aircraft  damage  by  exceeding  the  normal  operational  limits  (if  at  a  high 
thrust  setting),  or  end  in  a  dangerous  stall  (if  at  a  low  thrust  setting).  Although  each  aircraft  type 
presents  its  own  possible  scenarios  of  mode  error,  the  examples  provided  demonstrate  how  it 
might  be  possible  to  bridge  the  gap  between  theory  and  practice. 

The  terminology  in  Table  1  concerns  primarily  three  areas  of  the  cockpit  and  is  generally  C- 
32A  specific.  For  discussion  purposes  a  graphic  representation  of  the  forward  cockpit  is 
represented  in  Figure  3.  The  first  area  is  the  Mode  Control  Panel  (MCP)  located  in  the  center  of 
the  cockpit  dashboard.  It  is  where  the  pilot  sets  the  altitude,  aircraft  speed,  and  autopilot  mode 
functions.  Selections  on  this  panel  include  Altitude  (Alt)  Hold  which  tells  the  autopilot  to 
maintain  the  altitude  selected  on  the  MCP.1  Vertical  Navigation  (VNAV)  selects  a  vertical 
profile  calculated  by  the  Flight  Management  System  from  keypad  entries."  Flight  level  change 
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(FLCH)  uses  logic  based  on  aircraft  performance  to  change  current  altitude  to  that  selected  on 
the  MCP.  Vertical  Speed  (VSPD)  is  a  dial-in  function  that  sets  aircraft  climb  or  descent  to  a 
specific  foot-per-minute  rate.4  Lateral  Navigation  (LNAV)  performs  the  same  function  as  VNAV 
only  in  the  horizontal  plane.5  Approach  (APP)  mode  prepares  the  autopilot  to  intercept  a 
Glideslope  and/or  Localizer  on  an  approach  to  landing.6  The  GS  is  a  precision  beam  sent  by  a 
ground  station  that  vertically  guides  the  aircraft  down  to  landing.  The  LOC  is  a  beam  that 
horizontally  guides  an  aircraft  to  the  runway.  Together,  these  form  a  Precision  approach  that  is 
used  in  weather  so  poor  that  the  pilot  may  not  see  the  runway  until  only  seconds  from  landing. 
All  of  these  modes  engage  the  autothrottles  which  maintain  aircraft  speed  to  that  selected  on  the 
MCP.  Additionally,  the  MCP  has  an  Exhaust  Pressure  Ratio  (EPR)  mode  that  keeps  the  A/Ts  at  a 
particular  dial-in  thrust  setting. 
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Figure  3.  C-32A  Forward  Cockpit8 

The  second  area  of  the  cockpit  is  the  FMS  which  has  entry  keypads  located  on  both  sides  of 
the  center  control  stand.  It  is  a  complex  computerized  system  that  integrates  pilot  entered  course 
and  vertical  profile  information  (via  keypad),  MCP  inputs,  navigation  system  guidance,  and 
stored  database  values  to  control  the  autopilot.9  The  third  area  concerns  the  relative  position 
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displays  located  directly  in  front  of  both  pilots.  The  Attitude  Display  Indicator  (ADI)  presents 
aircraft  pitch  and  roll  relative  to  the  horizon.  Also  on  the  ADI  are  Flight  Management 
Annunciators  (FMA)  which  confirm  the  status  (armed,  engaged,  captured)  of  the  A/T,  autopilot, 
and  any  selected  modes.10  The  Horizontal  Display  Indicator  (HSI)  presents  horizontal  position 
relative  to  the  selected  FMS  course  or  ground  signal  (LOC).11 

The  scenarios  in  Table  1  are  presented  in  terms  of  the  automation  mode  selected,  a  possible 

corresponding  mode  error,  and  the  mitigating  audible  alert.  Appropriate  columns  describe  how 

an  alert  fulfills  the  three  principles  suggested  by  Human  Factors  studies  and  the  predictive  step 

suggested  by  other  research.  Techniques  that  might  intrude  into  the  pilot’s  cognitive  framework 

are  covered  under  the  “Attention”  column  by  volume,  sound  level,  and  number  of  repetitions. 

The  intensity  is  relative  to  existing  alerts,  with  “Loud”  representing  the  maximum  volume 

traditionally  used.  The  time  (seconds)  between  alerts  and  number  of  repetitions  was  strictly 

arbitrary  based  on  criticality  of  the  failure.  These  callouts  provide  situational  reporting  that  must 

be  different  from  any  other  alerts,  such  as  the  “glideslope”  warnings  provided  by  the  Ground 

12 

Proximity  Warning  System  (which  warns  of  excessive  deviation  from  the  GS  ground  signal). 
As  illustrated  in  the  third  and  fifth  examples,  adding  a  word  is  one  differentiation  method.  These 
callouts  are  easily  modified  since  software  controls  the  verbiage  and  intensity  of  voice  alerts  in 
most  modern  aircraft. 

Either  localizing  or  projecting  the  sound  with  a  small  number  of  well-placed  speakers 
(studies  suggest  six  or  less)  can  enhance  both  the  “Situation”  and  “Guidance”  principles.  Speaker 
location  is  fixed  since  the  pilot’s  seats  are  always  in  the  same  place  and  thus  will  have  the  same 
relative  effect  for  each  pilot.  The  predictive  function  could  be  tied  to  the  existing  computer  logic 
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used  by  the  FMS  to  calculate  course  and  altitude  intercepts  and  capture  zones.  These 
improvements  would  be  relatively  easy  to  implement  and  could  yield  tremendous  results. 


Table  1.  Notional  Examples  of  Mode  Error  Auditory  Alerts 


Automation  Status 

Alert  Principles 

Mode 

Error 

Attention 

Situation 

Guidance 

Predictive 

Alt  Hold 

Disengages 
without  pilot 
action 

Loud,  repeat 
every  five 
seconds  for 
five  times 

Callout 

“Check 

Altitude” 

Callout 
localized  to 
altimeter, 
MCP,  or 
FMA 

Immediate 

Alt  in 
MCP;  + 
VNAV, 
FLCH,  or 
VSPD 

Failure  to 
switch  to  Alt 
capture  mode 
within  preset 
logic 

Same  as  above 

Callout 

“Altitude 

Capture” 

Same  as 
above 

Callout  when  alt 
hits  normal 
capture  zone 
based  on  preset 
logic 

APP 

mode 

engaged 

Fails  to  switch 
to  GS  capture 
mode 

Same  as  above 

Callout 

“Glideslope 

Mode” 

Callout 
localized  to 
ADPFMAs 

Callout  when  in 
normal  capture 
zone  based  on 
good  GS  signal 
and  preset  logic 

APP  or 
(LOC) 
mode 
engaged 

Fails  to  switch 
to  LOC  capture 
mode 

Loud,  repeat 
every  three 
seconds  for 
five  times 

Callout 

“Localizer” 

Callout 
localized  to 
ADPHSI 
and  FMAs 

Same  as  above 
except  for  LOC 

APP, 
with  GS 
captured 

GS  disengaged 
without  pilot  or 
GS  signal  fails 

Same  as  above 

Callout 

“Glideslope 

Fail” 

Same  as 
above 

Position  exceeds 
preset  tolerance 
from  GS  beam 

APP  or 
LOC, 

LOC 

captured 

LOC 

disengaged 
without  pilot  or 
signal  fails 

Same  as  above 

Callout 

“Localizer 

Fail” 

Same  as 
above 

Position  exceeds 
preset  tolerance 
from  LOC  beam 

EPR, 

VNAV, 

FLCH 

A/T 

disengaged 
without  pilot 

Medium, 
repeat  every 
five  seconds 
for  five  times 

Callout 
“Auto  throttle 
off’ 

Same  as 
above 

Immediate 

LNAV 

Fails  to  switch 
to  FMS  course 
capture  mode 

Same  as  above 

Callout 

“Course” 

Same  as 
above 

Same  as  above 
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The  ideas  presented  and  synthesized  here  are  only  a  small  part  of  a  greater  effort  working  to 
understand  the  man-automation  interface  issue.  Although  much  emphasis  has  been  placed  on 
multi-seat  commercial-type  aircraft,  the  characteristics  of  mode  error  make  this  research  even 
more  applicable  to  complex  single-seat  aircraft  where  there  is  no  human  backup.  The  pilot- 
automation  interface,  types  and  consequences  of  mode  confusion,  cognitive  and  information 
modeling,  and  Human  Factors  sensory  studies  all  should  continue  as  each  plays  a  role  in  the  final 
answer.  In  the  larger  sphere,  improving  human-centered  design,  using  other  alerting  methods  and 
channels  (visual,  tactile?),  and  increasing  pilot  awareness  all  deserve  attention.  These  efforts 
could  possibly  discover  an  as  yet  unknown  approach  to  reducing  mode  error  or  its  consequences. 

Conclusion.  The  increasing  complexity  of  cockpit  automation  sometimes  results  in  mode 
confusion  that  might  be  mitigated  by  an  audible  alert.  The  SHEL  model  provides  a  basic 
framework  to  research  this  problem.  An  analysis  of  the  SHEL  Hardware  and  Software  arms 
shows  why  audible  alerts  are  needed.  Studies  reveal  that  a  great  majority  of  pilots  experience 
automation  surprises  whose  origins  were  not  immediately  understood,  if  comprehended  at  all. 
The  mode  confusion  that  occurs  falls  into  two  general  areas:  misunderstanding  of  designed 
active  or  passive  behavior;  and  misunderstanding  of  mode  error  (system  failure).  Increasing  pilot 
system  knowledge  might  reduce  the  former,  while  the  latter  might  be  mitigated  with  an 
appropriate  audible  alert.  Unfortunately,  current  systems  do  not  provide  the  type  of  feedback 
needed  to  make  these  failures  understandable.  Pilots  in  advanced  cockpits  make  automation 
related  errors  on  almost  every  flight.  These  errors  hold  the  potential  for  tragic  consequences. 
Accident  rates  have  remained  almost  constant  even  with  airline  procedural  standardization. 

One  method  of  mitigating  mode  error  is  by  introducing  an  audible  attention  step.  An 
analysis  of  the  Environment  and  Liveware  arms  clarifies  what  types  of  audible  alerts  would  be 
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most  effective.  Human  workload  and  processing  capacity  are  partially  dependent  on  the  amount 
of  information  and  time  available,  constituting  an  inversely  proportional  relationship  with 
performance.  Automation  itself  does  not  lower  this  workload  absolutely.  The  number  and  type  of 
systems  automated  as  well  as  the  extra  knowledge  needed  to  operate  and  monitor  the  automation 
(backup  mental  calculations  and  vigilance)  determine  the  true  workload  level.  Humans  also  do 
not  perform  monitoring  well  especially  over  time.  The  less  involved  the  operator,  the  lower  the 
system  situational  awareness,  and  the  harder  the  reorientation  in  case  of  malfunction. 

In  the  decision-making  process  pilots’  balance  risk  with  time.  Information  is  prioritized  as 
aviate,  navigate,  communicate,  or  system  administrate.  If  already  involved  in  one  task,  it  is 
difficult  to  divert  their  attention,  even  with  additional  alerting.  Pilots  also  tend  to  use  automation 
less  with  increasing  workload,  which  further  channels  their  attention.  This  situational  spectrum 
ranges  from  disengaged  boredom  to  cognitive  lockup.  Therefore,  a  predictive,  informational 
system  alert  that  demands  attention  and  intrudes  upon  the  pilot’s  cognitive  state  is  needed. 

The  predictive  function  should  address  the  workload  issue  and  provide  the  extra  time  needed 
for  a  situationally  aware  diagnosis.  The  informational  function  can  reduce  this  time  by  focusing 
the  pilot  on  the  correct  problem.  The  attention  aspect  is  addressed  in  the  three  principles  for 
effective  auditory  alerts  suggested  by  Human  Factors  studies.  First,  it  must  demand  attention  in 
order  to  cut  through  cognitive  lockup  or  task  focus.  It  should  be  reliable,  recognizable  above 
environmental  noise,  distinguishable  from  other  alerts,  and  not  be  duplicative  of  normal  pilot 
tasks.  Second,  it  should  provide  situation  reporting  with  short  understandable  messages 
commensurate  with  the  degree  of  urgency.  Lastly,  it  should  guide  the  pilot  to  the  problem 
mentally  and/or  literally.  A  predictive,  informational  alert  that  gets  a  distracted  pilot’s  attention 
can  effectively  reduce  the  recognition  and  reaction  time  for  mode  error. 
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Notes 


1  United  States  Air  Force  Document  D632N001-54  USF  (USF),  Boeing  757-264  Operating 
Manual,  3  vols.  (12  November  1999),  11.31.2. 

2  Ibid.,  11.31.16. 

3  Ibid.,  11.31.2. 

4  Ibid. 

5  Ibid.,  11.31.9. 

6  Ibid.,  4.20.11. 

7  Ibid. 

8  Ibid.,  NP.  10.4. 

9  Ibid.,  11.31.1. 

10  Ibid.,  4.10.3. 

11  Ibid.,  11.31.16. 

12  Ibid.,  15.10.10. 
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